Perf bwd hip #23
Open
Perf bwd hip #23
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
split_tbe_bwd.hip.cpp needed a hpp counterpart to invoke the necessary macros to build all templates for split_tbe_bwd_hip_kernel_
dllehr-amd
reviewed
Nov 11, 2022
| @@ -8,6 +8,10 @@ | |||
| {% set wdesc = "weighted" if weighted else "unweighted" %} | |||
| #include "fbgemm_gpu/embedding_backward_template_helpers.cuh" | |||
| #include "fbgemm_gpu/split_embeddings_utils.cuh" | |||
| #include "hip_kernel/split_tbe_common_hip.h" | |||
There was a problem hiding this comment.
Use
#ifdef HIP_PLATFORM_HCC
#endif
Around new includes
dllehr-amd
reviewed
Nov 11, 2022
| @@ -45,6 +45,7 @@ | |||
| # An optimization for ROCm | |||
| env.globals["items_per_warp"] = 128 if args.is_rocm is False else 256 | |||
There was a problem hiding this comment.
If we have use_rocm being exposed to jinja, we can probably avoid needing an extra "items_per_warp"
dllehr-amd
reviewed
Nov 11, 2022
| static hipFunction_t hip_kernel_func; | ||
| {% if optimizer == "rowwise_adagrad" and not dense %} | ||
| std::set<int> D_emb_s {64, 128, 192, 256}; | ||
| bool hip_opt_kernel_supported = (D_emb_s.find(max_D) != D_emb_s.end()) && |
There was a problem hiding this comment.
When we do mixed dimension this check will go away.
liligwu
pushed a commit
that referenced
this pull request
Feb 8, 2023
IFU targets on upstream 5219dc4
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
python test/split_table_batched_embeddings_test.py SplitTableBatchedEmbeddingsTest.test_backward_adagrad_fp32_pmSUMpython test/split_table_batched_embeddings_test.py SplitTableBatchedEmbeddingsTest.test_backward_optimizers_adagradCan pass above 2 UTs ( by some modification)
emb_tandgrad_tcombinationexactfor now.weight_decay_modein rowwise-adagradtest_backward_adagrad_fp32_pmSUM)